SMML estimators for exponential families with continuous sufficient statistics

The minimum message length principle is an information theoretic criterion that links data compression with statistical inference. This paper studies the strict minimum message length (SMML) estimator for d-dimensional exponential families with continuous sufficient statistics, for all d. The partition of an SMML estimator is shown to consist of convex polytopes (i.e. convex polygons when d = 2). A simple and explicit description of these polytopes is given in terms of the assertions and coding probabilities, namely that the i polytope is exactly the set of data points where the posterior probability corresponding to the i assertion and i coding probability is greater than or equal to the other posteriors. SMML estimators which partition the data space into n regions are therefore determined by n(d + 1) numbers which describe the assertions and coding probabilities, and we give n(d+1) equations that these numbers must satisfy. Solving these equations with a quasiNewton method then gives a practical method for constructing higher-dimensional SMML estimators.